Ensemble Confidence Estimates Posterior Probability
نویسندگان
چکیده
We have previously introduced the Learn algorithm that provides surprisingly promising performance for incremental learning as well as data fusion applications. In this contribution we show that the algorithm can also be used to estimate the posterior probability, or the confidence of its decision on each test instance. On three increasingly difficult tests that are specifically designed to compare posterior probability estimates of the algorithm to that of the optimal Bayes classifier, we have observed that estimated posterior probability approaches to that of the Bayes classifier as the number of classifiers in the ensemble increase. This satisfying and intuitively expected outcome shows that ensemble systems can also be used to estimate confidence of their output.
منابع مشابه
Empirical Bayes interval estimates that are conditionally equal to unadjusted confidence intervals or to default prior credibility intervals.
Problems involving thousands of null hypotheses have been addressed by estimating the local false discovery rate (LFDR). A previous LFDR approach to reporting point and interval estimates of an effect-size parameter uses an estimate of the prior distribution of the parameter conditional on the alternative hypothesis. That estimated prior is often unreliable, and yet strongly influences the post...
متن کاملCombining independent Bayesian posteriors into a confidence distribution, with application to estimating climate sensitivity
Combining estimates for a fixed but unknown parameter to obtain a better estimate is an important problem, but even for independent estimates not straightforward where they involve different experimental characteristics. The problem considered here is the case where two such estimates can each be well represented by a probability density function (PDF) for the ratio of two normally-distributed ...
متن کاملEnsemble Learning for Hidden Markov Models
The standard method for training Hidden Markov Models optimizes a point estimate of the model parameters. This estimate, which can be viewed as the maximum of a posterior probability density over the model parameters, may be susceptible to over-tting, and contains no indication of parameter uncertainty. Also, this maximummay be unrepresentative of the posterior probability distribution. In this...
متن کاملImproved Gaussian Mixture Density Estimates Using Bayesian Penalty Terms and Network Averaging
Volker Tresp Siemens AG Central Research 81730 Munchen, Germany Volker. [email protected] We compare two regularization methods which can be used to improve the generalization capabilities of Gaussian mixture density estimates. The first method uses a Bayesian prior on the parameter space. We derive EM (Expectation Maximization) update rules which maximize the a posterior parameter probabili...
متن کاملComparison of the Bayesian and Randomised Decision Tree Ensembles within an Uncertainty Envelope Technique
Multiple Classifier Systems (MCSs) allow evaluation of the uncertainty of classification outcomes that is of crucial importance for safety critical applications. The uncertainty of classification is determined by a trade-off between the amount of data available for training, the classifier diversity and the required performance. The interpretability of MCSs can also give useful information for ...
متن کامل